Galois Connections Between Semimodules and Applications in Data Mining

In [1] a generalisation of Formal Concept Analysis was introduced with data mining applications in mind, K-Formal Concept Analysis, where incidences take values in certain kinds of semirings, instead of the standard Boolean carrier set. A fundamental result was missing there, namely the second half of the equivalent of the main theorem of Formal Concept Analysis. In this continuation we introduce the structural lattice of such generalised contexts, providing a limited equivalent to the main theorem of K-Formal Concept Analysis which allows to interpret the standard version as a privileged case in yet another direction. We motivate our results by providing instances of their use to analyse the confusion matrices of multiple-input multiple-output classifiers. 1 Motivation: the Exploration of Confusion Matrices with K-Formal Concept Analysis In pattern recognition tasks, when a classifier is provided training data in the form of feature vectors tagged with an input pattern set and produces for each vector a tag within an output pattern set, the performance of the classifier can be gleaned from the collection of pairs (gi,mj) of one input tag, gi, for the input data and one output tag, mj , produced by the classifier. These results are aggregated into a confusion matrix, T , whose element Tij gives a “measure” of the joint event (G = gi,M = mj), “providing an input pattern gi to the classifier who then produces an output pattern mj”. In the pattern recognition community we often encounter methods that use confusion matrices to analyse classification results. However, most of the times the analysis is manual and limited to the (human-based) pondering of a confusion matrix-representation like the one depicted in figure 1, where the warmer, brighter (resp. cooler, darker) colour hues are designed to be related to high occurrence (resp. to low occurrence) of events. Often, this type of analysis is used ? This work has been partially supported by two grants for “Estancias de Tecnólogos Españoles en el International Computer Science Institute, año 2006” of the Spanish Ministry of Industry and a Spanish Government-Comisión Interministerial de Ciencia y Tecnoloǵıa project TEC2005-04264/TCM. Fig. 1. Confusion matrix of the desired transformation of English phoneme labels of speech frames versus their true Mandarin phoneme labels to bootstrap existing classifiers in order to obtain even better classification figures or simply to understand the underlying principles of the methods employed in designing the classification. In particular, in speech recognition, the designer of a system is challenged to find in this type of representation meaningful or systematic confusions to determine to what degree the behaviour of an automatic system differs from human performance. K-Formal Concept Analysis was introduced in [1] as a generalisation of standard Formal Concept Analysis in the sense that incidences R ∈ Kn×p represented as matrices may take values in an idempotent, reflexive semifield K and we take R(i, j) = λ to mean “object gi has attribute mj in degree λ.” Adequate analogues of basic objects in Formal Concept Analysis become therefore available. Two serious obstacles may prevent widespread adoption of K-Formal Concept Analysis as a data exploration technique complementary to the standard theory: on the one hand, the K-Formal Concept Analysis analogue of the main theorem of Formal Concept Analysis is incomplete and this may worry the user willing to be on a sound mathematical ground; on the other hand, [1] did not provide an algorithm for constructing the lattice of a K-valued formal context, which prevents its use as a data-intensive exploration procedure. In this paper, we try to explore further whether K-Formal Concept Analysis is a proper generalisation of standard Formal Concept Analysis for finite contexts and to pave the way for the completion of the main theorem. In order to do so we introduce the structural lattice of a K-Formal Context and try to relate it to the Concept Lattice of a Formal Context. In section 2 we first review the theory of idempotent semirings and their semimodules with a view to providing the necessary objects for our discussion. In section 3.1 we present a summary of the theory of K-Formal Concept Analysis presented in ([1], §. 3) and add a new theoretical construct, the structural lattice of a semimodule over an idempotent, reflexive semiring. We demonstrate in section 4 the use of this new tool to analyse confusion matrices of multiple input-multiple output classifiers, which turn out to be amenable to K-Formal Concept Analysis modelling, and finish with a summary of contributions and an outlook. 2 Mathematical Preliminaries: semimodules over idempotent, reflexive semifields as vector spaces 2.1 Idempotent Semirings A semiring K = 〈K,⊕,⊗, , e〉 is an algebraic structure whose additive structure, 〈K,⊕, 〉, is a commutative monoid and the multiplicative one, 〈K,⊗, e〉, a monoid whose multiplication distributes over addition from right and left and whose neutral element is absorbing for ⊗, ⊗ x = ,∀x ∈ K [2] . On any semiring K left and right multiplications can be defined: La : K → K Ra : K → K (1) b 7→ La(b) = ab b 7→ Ra(b) = ba A commutative semiring is a semiring whose multiplicative structure is commutative, and a semifield one whose multiplicative structure over K\{ } is a group. Thus, compared to a ring, a semiring which is not a ring lacks additive inverses. An idempotent semiring K is a semiring whose addition is idempotent: ∀a ∈ K, a ⊕ a = a . All idempotent commutative monoids (K,⊕, ) are endowed with a natural order ∀a, b ∈ K, a ≤ b ⇐⇒ a ⊕ b = b , which turns them into join-semilattices with least upper bound defined as a∨ b = a⊕ b . Moreover, for the additive structure of and idempotent semiring K the neutral element is the infimum for this natural order, K = ⊥ . An idempotent semiring K is complete, if it is complete as a naturally ordered set and left (La) and right (Ra) multiplications are lower semicontinuous, that is, they commute with joins over any subset of K . Therefore, complete idempotent semirings, as join-semilattices with infimum are automatically complete lattices [3] with join (∨, max or sup) and meet (∧, min or inf) connected by the equivalences: ∀a, b ∈ K, a ≤ b ⇐⇒ a ∨ b = b ⇐⇒ a ∧ b = a . Example 1. 1. The Boolean semiring B = 〈B,∨,∧, 0, 1 〉, with B = {0, 1} , is complete, idempotent and commutative. 2. The completed Maxplus semiring Rmax,+ = 〈R ∪ {±∞},max,+,−∞, 0 〉 , is a complete, idempotent semifield when defining −∞ +∞ = −∞, so that K ⊗>K = K for K ≡ Rmax,+ 3. The completed Minplus semiring Rmin,+ = 〈R ∪ {±∞},min,+,∞, 0 〉 is a complete, idempotent semifield with a similar completion to that of ex. 2 with ∞+ (−∞) =∞, that is K ⊗>K = K for K ≡ Rmin,+ . 2.2 Idempotent Semimodules: Basic Definitions A semimodule over a semiring is defined in a similar way to a module over a ring [4,5,6] : a left K-semimodule, X = 〈X,⊕, X〉, is an additive commutative 1 We are following essentially the notation of [4]. monoid endowed with a map (λ, x) 7→ λ · x such that for all λ, μ ∈ K, x, z ∈ X, and following the convention of dropping the symbol for the scalar action and multiplication for the semiring we have: (λμ)x = λ(μx) Kx = X (2) λ(x⊕ z) = λx⊕ λz eKx = x The definition of a right K-semimodule, Y, follows the same pattern with the help of a right action, (λ, y) 7→ yλ and similar axioms to those of (2.) A (K,S)-semimodule is a set M endowed with left K-semimodule and a right Ssemimodule structures, and a (K,S)-bisemimodule a (K,S)-semimodule such that the left and right multiplications commute. For a left K-semimodule, X , the left and right multiplications are defined as: Lλ : X → X RX x : K → X (3) x 7→ Lλ (x) = λx λ 7→ RX x (λ) = λx And similarly, for a right K-semimodule. If X , Z are left semimodules a morphism of left semimodules or left linear map F : X → Z is a map that preserves finite sums and commutes with the action: F (λv ⊕ μw) = λF (v)⊕ μF (w), and similarly, mutatis mutandis for right linear maps of right semimodules. The elements of a semimodule may be conceived as vectors. Given a semiring K and a left K-semimodule X , for each finite, non-void set W ⊆ X, there exists an homomorphism α : K → X, f 7→ ⊕ w∈W f(w)w . Moreover, α induces a congruence of semimodules ≡α on K , by f ≡α g ⇐⇒ α(f) = α(g) . Then W is a set of generators or a generating family precisely when α is surjective, in which case any element x ∈ X can be written as x = ⊕ w∈W λww, and we will write X = 〈W 〉K, that is, X is the span of W . A semimodule is finitely generated if it has a finite set of generators. For individual vectors, we say that x ∈ W is dependent (in W ) if x = ⊕ w∈W\{ x } λww otherwise, we say that it is free (in W ). The set W is linearly independent if and only if ≡α is the trivial congruence, that is, when ⊕ w∈W f(w)w = ⊕ w∈W h(w)w ⇐⇒ f = h, otherwise, W is linearly dependent. Let kerα = { f ∈ K | α(f) = 0 }; then W is weakly linearly independent if and only if kerα = {0}, otherwise it is weakly linearly dependent. A basis for X (over K) is a linearly-independent set of generators, and a semimodule generated by a basis is free. By definition, in a free semimodule X with with basis {xi }i∈I each element x ∈ X can be uniquely written as x = ⊕ i∈I αixi, with [ai]i∈I the co-ordinates of x with respect to the basis. A weakly linearly-independent set of generators for X is a weak basis for X (over K). The cardinality of a (weak) basis is the (weak) rank of the semimodule. In this framework, notions in usual vector spaces have to be imported with care. For instance, the image of a linear map F : X → Y is simply the semimodule ImF = {F (x) | x ∈ X }, but it is in general not free. 2 Most of the material in this section is from [5], §17, and [7,8,9]. Given a free semimodule X with basis {xi }i∈I , for each family { yi }i∈I of elements of an arbitrary semimodule Y there is a unique morphism of semimodules F : X → Y such that F (xi) = yi,∀i ∈ I, namely F (⊕ i∈I λixi ) = ⊕ i∈I λiyi and all the linear maps Lin(X ,Y) are obtained in this way ([7], prop. §73; [5], prop. §17.12). That is, linear maps from free semimodules are characterised by the images of the elements of a basis. On the other hand, a semiring K has the linear extension property if for all free, finitely generated K-semimodules X ,Y, for all finitely generated subsemimodules Z ⊂ X and for all F ∈ Lin(Z,Y), there exists H ∈ Lin(X ,Y) such that ∀x ∈ X,H(x) = F (x) . The importance of this property derives from the fact that when the linear extension property holds, each linear map between finitely generated subsemimodules of free semimodules is represented by a matrix. In particular, when it holds for free, finitely generated (left) semimodules, X and Y with bases {xi }i∈I and { yj }j∈J , each linear map is characterised by the n × p-matrix R = (F (xi)j), which sends vector x = {xi}i=1 to the vector F (x) ' ((xR)1, . . . , (xR)p) . 2.3 Semimodules over Idempotent Semirings In this section all semimodules will be defined over an idempotent semifield. Recall that examples of these are B, the Boolean semifield and the completed maxplus and minplus semifields. Idempotency and Natural Order in Semimodules. A left, right K-semimodule X over an idempotent semiring K inherits the idempotent law, v ⊕ v = v,∀v ∈ X, which induces a natural order on the semimodule by v ≤ w ⇐⇒ v ⊕ w = w,∀v, w ∈ X whereby it becomes a ∨-semilattice, with X the minimum. In the following we systematically equate idempotent K-semimodules and semimodules over an idempotent semiring K . When K is a complete idempotent semiring, a left K-semimodule, X is complete (in its natural order) if it is complete as a naturally ordered set and its left and right multiplications are (lower semi)continuous. Trivially, it is also a complete lattice, with join and meet operations given by: v ≤ w ⇐⇒ v ∨ w = w ⇐⇒ v ∧ w = v . This extends naturally to rightand bisemimodules. Example 2. 1. Each semiring, K, is a left (right) semimodule over itself, with the semiring product as left (right) action. Therefore, it is a (K, K)bisemimodule over itself, because both actions commute by associativity. Such is the case for the Boolean (B,B)-bisemimodule, the Maxplus and the Minplus bisemimodules. These are all complete and idempotent. 2. For n,m ∈ N, the set of matrices Kn×p is a (Kn×n,Kp×p)-bisemimodule with matrix multiplication-like left and right actions and component-wise addition, the set of column vectors Kp×1 is a (Kp×p,K)-bisemimodule and the set of row vectors K1×n a (K,Kn×n)-bisemimodule with similarly defined operations. If K is idempotent (resp. complete), then all are idempotent (resp. complete) with the component-wise partial order their natural order. As in the semiring case, because of the natural order structure, the actions of idempotent semimodules admit residuation: given a complete, idempotent left K-semimodule, X , we define for all x, z ∈ X, λ ∈ K the residuals: ( Lλ )# : X → X ( Lλ )# (z) = ∨ {x ∈ X | λx ≤ z } = λ\z (4) ( RX x )# : X → K ( RX x )# (z) = ∨ {λ ∈ K | λx ≤ z } = z/x and likewise for a right semimodule, Y . There is a remarkable operation that changes the character of a semimodule while at the same time reversing its order by means of residuation: Definition 3. Let K be a complete, idempotent semiring, and Y be a complete right K-semimodule, its opposite semimodule is the complete left K-semimodule Y = 〈Y, op ⊕, op →〉 with the same underlying set Y , addition defined by (x, y) 7→ x op ⊕ y = x ∧ y where the infimum is for the natural order of Y, and left action: K × Y → Y (λ, y) 7→ λ op → y = y/λ Consequently, the order of the opposite is the dual of the original order. For the opposite semimodule the residual definitions are: λ op \ x = ( LY op λ )# (x) = ∧ { y ∈ Y | x ≤ y/λ } = x · λ (5) x op / y = ( RY op y )# (x) = ∨ {λ ∈ K | x ≤ y/λ } = x\y Note that we can define mutatis mutandis the opposite semimodule of a left K-semimodule, X , with right action x op ← λ = λ\x . Also, noticing that the first residual in eq. 5 is in fact an involution we may conclude that the operation of finding the opposite of a complete (left, right) K-semimodule is an involution: (Yop) = Y . Constructing Galois Connections in Idempotent Semimodules. The following construction is due to Cohen et al. [4]. Let K be a complete idempotent semiring; for a bracket 〈· | ·〉 : X×Y → Z between left and right K-semimodules, X and Y respectively, onto a K-bisemimodule Z and an arbitrary element φ ∈ Z, which we call the pivot, define the maps: ·φ : X → Y xφ = Lx (φ) = ∨ { y ∈ Y | 〈x | y〉 ≤ φ } (6) − φ · : Y → X φ y = R y (φ) = ∨ {x ∈ X | 〈x | y〉 ≤ φ } We have 〈x | y〉 ≤ φ ⇐⇒ y ≤ xφ ⇐⇒ x ≤ φ y, whence the pair is a Galois connection between Y and X , (·φ ,φ ·) : X ( Y . This construction is affected crucially by the choice of a suitable pivot φ: if we consider the bracket to reflect a degree of relatedness between the elements of each pair, only those pairs (x, y) ∈ X × Y are considered by the connection whose degree amounts at most to φ . Therefore we can think of the pivot as a maximum degree of existence allowed for the pairs. Recall X and Y are both complete lattices as well as free vector spaces. Note that the closure lattices X = φ (Y) and Y = (X )φ do not agree with their ambient vector spaces in their joins, but only in their meets. To improve on this, the notion of a left (resp. right) reflexive, (K, φ), semiring is introduced in [4] as a complete idempotent semiring such that (〈· | ·〉 : K × K → K,φ) with 〈λ | μ〉 = λμ induces a perfect Galois connection under construction (6) for all λ ∈ K, −(λ−) = λ (resp. (−λ)− = λ .) The interest in reflexive semirings stems from the fact that in such semirings X and Y are actually subsemimodules (that is their suprema coincide with those) of the corresponding spaces ([4], prop. 28). Note that φ need not be unique: if (K, φ) is right (or left) reflexive, for any λ ∈ K invertible, (K, φλ) is left reflexive (and (K, λφ) is right reflexive.) Finally, Cohen et al. [4] prove that idempotent semifields are left and right reflexive, and suggest that for the Boolean semiring we must choose φ = 0B, the bottom in the order. For other semifields any invertible element may be chosen, e.g. φ = eK . Idempotent Semimodules as Vector Spaces. When K is an idempotent semiring if a K-semimodule has a (weak) basis, it is unique up to a permutation and re-scaling of the axes, that is a scaling endomorphism ([9], Th. §3.1), xi = λixi , and every finitely generated K-semimodule has a weak basis ([9], Coroll. §3.6). In particular, let K be an idempotent semifield, then the free idempotent semimodule with n generators is isomorphic to K . Essentially, such free idempotent semimodules are generated by the bases En , {ei}ni=1 , ei = (δi1, δi2, . . . , δin), where δij is the Kronecker symbol over K, δii = eK, δij = K, i 6= j . Importantly, the linear property holds in every idempotent semiring which is a distributive lattice for the natural order ([7], Th. §83). This is the case for the semifields B (the Boolean semiring), Rmax,+ and Rmin,+ . Therefore, in such semimodules, modulo a choice of bases for X and Y, we may identify X ∼= K1×n and Y ∼= K1×p, and linear maps to matrix transformations Lin(X ,Y) ∼= Kn×p, R : K1×n → K1×p, x 7→ xR . When passing from left to right semimodules this should read Kp×1 → Kn×1, y 7→ Ry . Idempotent semimodules have additional properties which make them easier to work with as spaces: when X is a vector space over an idempotent semiring K, for a set of vectors, W ⊆ X , the set of finite sums W , { ⊕ i wi | wi ∈W}, is a ∨-subsemilattice of 〈W 〉K . Therefore, the ∨-irreducibles of W , generate the span of W , 〈J (W )〉K = 〈W 〉K . This makes the ∨-irreducibles an interesting set to obtain a basis. 3 That is, a pair of mutually inverse isomorphisms. 4 When the pivot is the multiplicative unit φ = e we drop it. The Projective Space and the Structural Semilattice. Let X be a left K-semimodule over an idempotent semiring K. The relation x 4 y ⇔ ∃λ ∈ K,x ≤ λ⊗ y defines a quasi-order 〈X ,4〉 . Since any basis WX is unique up to a re-scaling map, the Hasse diagram of (WX ,4) is independent of the choice of basis. Now define the equivalence relation ([7], p. 41), x ' y 4 ⇔ x 4 y and y 4 x . This relation appears already in ([10], p. 2018) and was later considered under the name of siblinghood relation [11] where two vectors v and w are siblings if w = λ⊗ v for some λ ∈ K . This is a congruence of ∨-semilattices, therefore [7], the projective space is the quotient set P(X ) , {[x]' | x ∈ X} (where [x]' the equivalence class of x ∈ X , is also called the ray of x or the sibling class of x ), which is also a ∨-semilattice, 〈P(X ),4〉 with the induced order. For any subset W ⊆ X, let a section of the quotient set W/' , σ : 2 → X,W 7→ σ(W ) be a set obtained by choosing a single representative from each sibling class. Note that a section has the order directly induced by 4X [11] . It is now clear that a section of the quotient set of the join irreducibles of a set of vectors is a (weak) basis of their span σ [J (W )] = 〈W 〉K . Next, consider the siblinghood relation above and a basis WX : Definition 4 (Wagneur [10]). Let X be a left K-semimodule over and idempotent semifield K with a basis WX . The structural (∨-)semilattice of X , S(X ) is the quotient set of WX through the siblinghood relation S(X ) , WX/' . The following theorem states that the quotient set WX/' is an intrinsic invariant of X . Theorem 1 ([10], Th. 2). For any basis WX of a left K-semimodule over and idempotent semifield K, the quotient map π : WX → WX/', w 7→ [w]' is an epimorphism of ∨-semilattices and WX/' is independent of the particular choice of basis WX . Since π is an epimorphism of ∨-semilattices and ' a ∨-congruence, the quotient set of the basis through the siblinghood relationWX /' = {[w]' | w ∈WX } is the set of ∨-irreducibles of the quotient set, J ( WX /' ) =WX /' [10]. 3 The Structural Lattice of a K-Concept Lattice 3.1 K-Formal Concept Analysis, a Reminder The following has been adapted from [1] to emphasise the fact that the theory does not cover the case of unbounded cardinalities. Definition 5 (K-valued formal context ). For n, p ∈ N, given two sets of objects G = {gi}i=1, and attributes M = {mj} p j=1, an idempotent semiring, K, and a K-valued matrix, R ∈ Kn×p, where R(i, j) = λ reads as “object gi has attribute mj in degree λ” and dually “attribute mj is manifested in object gi to degree λ”, the triple (G,M,R)K is called a K-valued formal context. 5 This section follows in the tracks of §1.1 of [12]. Clearly single objects are isomorphic to elements of the space K1×p, that is rows of R or object descriptions, vectors of as many values as attributes. And dually, single attributes are isomorphic to elements of the space Kn×1, columns of R or attribute descriptions. We model (K-valued) sets of objects as row vectors in a left K-semimodule, x ∈ X ∼= K1×n, and sets of attributes as column vectors in a right K-semimodule, y ∈ Y ∼= Kp×1 as generalisations of characteristic functions in the power sets 2,2 , respectively. The proof of the following proposition is crucial for future argumentation, hence we reproduce it in full: Proposition 2. Let (K, φ) be a reflexive, idempotent semiring. For a K-valued formal context (G,M,R)K, with n, p ∈ N, there is at least one Galois connection between the lattices of (K-valued) sets of objects K1×n and attributes Kp×1 . Proof. Recall that X = K1×n is a left semimodule and Y = Kp×1 a right semimodule, whence X op and Y are right and left semimodules, respectively, whose multiplications are R op ← x = x\R and y op → R = R/y . We build a new bracket over the opposite semiring K as given by 〈y | x〉R = y op → R op ← x = x\R/y. Therefore, by the construction of section 2.3 the following maps form a Galois connection (·φ ,φ ·) : Y ( X op : y− φ = ∧ {x ∈ X | 〈y | x〉R≥φ } = ( y op → R ) op \ φ (7) − φx = ∧ { y ∈ Y | 〈y | x〉R≥φ } = φ op